Mathematical Frameworks for Pricing in the Cloud: 
Revenue, Fairness, and Resource Allocations 



Carlee Joe-Wong Soumya Sen 

Princeton University 
{cjoe,soumyas}@princeton.edu 



ABSTRACT 

As more and more users begin to use the cloud for their com- 
puting needs, datacenter operators are increasingly pressed 
to eflFectively allocate their resources among these client users. 
Yet while much work has been done in this area, relatively 
little attention has been paid to studying perhaps the ulti- 
mate lever of resource allocation: pricing. Most data centers 
today charge users by "bundling" heterogeneous resources 
together in a fixed ratio and selling these bundles to their 
clients. But bundling masks the fact that different users re- 
quire different combinations of resources (e.g., CPUs, mem- 
ory, bandwidth) to process their jobs. The presence of mul- 
tiple resources in fact allows an operator to offer many differ- 
ent types of pricing strategies, which may have different ef- 
fects on its revenue. Moreover, to avoid user dissatisfaction, 
operators must consider the impact of their chosen prices on 
the fairness of the jobs processed for different users. In this 
paper, we develop an analytical framework that accounts for 
the fairness and revenue tradeoffs that arise in a datacenter's 
multi-resource setting and the impact that different pricing 
plans can have on this tradeoff. We characterize the im- 
plications of different pricing plans on various fairness met- 
rics and derive analytical limits on the operator's fairness- 
revenue tradeoff. We then provide an algorithm to navigate 
this tradeoff and compare the tradeoff points for different 
pricing strategies on a data trace taken from a Google clus- 
ter. 

Categories and Subject Descriptors 

K.6.2 [Management of Computing and Information 
Systems]: Installation Management — Pricing and resource 
allocation 

General Terms 

Economics, Management 
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1. INTRODUCTION 

1.1 Cloud Computing Resources 

As Internet connectivity becomes ever more ubiquitous, 
cloud computing is becoming an increasingly popular alter- 
native to traditional computing paradigms [26]. This dra- 
matic increase in demand has led to several research chal- 
lenges: how should a datacenter allocate multiple shared 
resources among users in a practical, scalable manner? The 
research implications of this question range from develop- 
ing an efficient method of routing client requests to different 
datacenters, to allocating a certain amount of resources for 
processing each job as it arrives at a particular datacenter. 
In this work, we focus on an "infrastructure-as-a-service" 
view of the cloud, in which the resources considered are raw 
computing resources such as CPU time or memory. 

When provisioning resources to users, datacenter opera- 
tors must take into account the fact that multiple types of 
resources are being allocated. For instance, users' jobs re- 
quire both CPU time and RAM, which are non-substitutable 
resources: more RAM does not mitigate a need for CPU pro- 
cessing time, and neither does more processing time signifi- 
cantly lessen RAM requirements. This provisioning of multi- 
ple resources thus introduces new challenges to the datacen- 
ter operator's resource allocation problem, which are only 
beginning to be studied [TT]. In this work, we consider the 
consequences of multiple resource allocation on the prices 
offered to datacenter users. 

1.2 Pricing Objectives 

As datacenters become ever more popular, many position 
papers have argued for the need to research new pricing 
mechanisms that can accommodate the competing objec- 
tives of a datacenter operator [4] [26]. While the operator 
of course wishes to maximize its revenue, its pricing policy 
must also take into account the dynamic nature of user de- 
mand: as users submit jobs to the datacenter, the operator 
must be able to accommodate the influx of new jobs and 
balance the needs of incoming users with those of users al- 
ready in queue. We thus introduce a fairness dimension: to 
avoid incurring user dissatisfaction, the datacenter operator 
should allocate its resources so that each user can process 
a fair, or equitable, number of jobs. Treating users fairly, 
however, can in itself impact revenue. In this work, we in- 
troduce an optimization framework that allows datacenter 
operators to balance the revenue and fairness of its resource 
allocation through setting prices for its computing resources. 

The presence of multiple resources gives a datacenter op- 



erator flexibility in offering different types of pricing plans. 
In this work, we examine the implications of different pricing 
schemes on the achieved fairness and revenue. We consider 
three different pricing plans: 

Bundled pricing: Bundled pricing assumes that opera- 
tors group heterogeneous resources such as CPU time and 
memory together and then sell this resource bundle to users. 
For instance, Google's Compute Engine sells virtual ma- 
chines (VMs), each with a fixed CPU and memory capacity 
[7]. Basic pricing policies for Amazon's EC2 similarly offer 
VM bundles [2]. Users are charged at a fixed unit rate per 
VM; they are free to make use of all or part of each VM re- 
source. For instance, a user with computationally intensive 
jobs might not require the VM's full RAM capacity, but will 
need to purchase its use anyway. 

Resource pricing: Under resource pricing, an opera- 
tor charges users separate unit prices for each resource that 
they use. Microsoft's Windows Azure, for instance, charges 
separate unit rates for CPU time and memory [TJ. Unlike 
bundled pricing, resource pricing does not charge users for 
resources they do not use. One can, however, view resource 
pricing as a form of bundled pricing, in which the bundles for 
each user are customized to exactly fit that user's resource 
requirements. The price of each bundle is determined by the 
unit prices of the different resources. 

Differentiated pricing: In differentiated pricing, users 
are allocated exactly as much of each resource as they re- 
quire to process their jobs. Differentiated pricing differs from 
resource pricing, however, in that the operator can freely de- 
termine each user's per-job cost, and is not constrained to 
charge users by unit resource prices. While differentiated 
pricing has not yet been introduced to the market, many 
believe that it will be necessary in the future [16] . 

In addition to the three pricing plans introduced above, 
one may take advantage of datacenters' economies of scale by 
introducing volume discounts into a pricing policy; Ama- 
zon's EC2 offers such a pricing plan, charging a lower unit 
rate for users who purchase a large quantity of resources [2] . 
While some works have investigated the effects of volume 
discounts, e.g., for contract users [3], none have done so in a 
multi-resource context. In this work, we compare bundled, 
resource, and differentiated pricing with volume discounts 
in a mathematical framework that takes into account user 
demand, operator revenue, and fairness across users. 

While the pricing plans offered by specific operators have 
been analyzed and compared [18II23| . we use a general model 
to solve for an operator's optimal prices. Our work is unique 
in accounting for both the pricing of multiple resources and 
the fairness of the resulting allocation. While other papers 
have accounted for fairness by using game theory to model 
user competition [19] or studying the strategy-proof prop- 
erty [20], these works do not explicitly deal with the pres- 
ence of multiple, non-substitutable resources. Works that 
do consider multi-resource fairness, on the other hand, do 
not include user pricing. Instead, operators are assumed to 
allocate a certain number of jobs to users in a manner that is 
both efficient, in terms of a large number of processed jobs, 
and equitable, i.e., no user processes a disproportionately 
high or low number of jobs [6l 1111 [22] . 

1.3 Job Arrivals and Deadlines 

Dynamic pricing has been perhaps the most common ap- 
proach taken in the literature on pricing for cloud or data- 



center computing. Some papers have investigated revenue- 
maximizing prices [5] [27] , while other models take into ac- 
count such factors as job completion deadlines [24]. In an- 
other variant, users' willingness to pay and job prices vary 
depending on a given job's completion time [21]. For in- 
stance, operators may give different price quotes to users as 
a function of the completion time, and the user can select 
from these quotes [lO]. Such a formulation may be used to 
maximize the long-term social surplus, defined in terms of 
user utilities over time [13]. However, most of these works 
do not take the presence of multiple resources into account. 
Other works have proposed dynamic auctions, an approach 
popular with grid computing [5] [17]. However, such mea- 
sures, e.g., the spot pricing offered by Amazon's EC2, have 
been shown to be ineffective in practice [25j . 

In our work, we suppose that the datacenter operator op- 
timizes its prices over a finite time horizon. We divide time 
into discrete intervals, e.g., of one hour, and suppose that 
users are allocated the use of a given resource (e.g., CPU) 
for each particular interval|3 We then have two possibilities: 
in the first, user demands could be the same during each 
time interval. The operator can then compute its prices in 
an offiine optimization and simply offer those prices during 
each interval. In the second, more realistic, scenario, users' 
demands change over time, and the operator must change 
its prices accordingly. Operators must then account for job 
deadlines, so that users' jobs complete in a time acceptable 
to users. In the main body of this work, we consider the 
first scenario of optimization in a single time interval; Ap- 
pendix |X] shows how our work may be extended to account 
for changes in user demand and job deadlines. 

We begin developing our framework in Section [2] by ex- 
amining users' choices of how many jobs to submit, given 
a set of offered prices. Section [3] then discusses bundled, 
resource, and differentiated pricing in the context of users' 
choices and derives expressions for the operator's revenue. 
In Section|4] we introduce a family of metrics to measure the 
fairness under different pricing plans. We then characterize 
the fairness-revenue tradeoff with these metrics and derive 
algorithms for practically choosing prices at different points 
on the tradeoff. Finally, in Section[5]we perform simulations 
on a dataset from a Google cluster to compare the achieved 
fairness and efficiency for different pricing plans and a range 
of resource capacities and volume discounts. Section [6] con- 
cludes the paper. 

2. USER OPTIMIZATION 

Suppose that a datacenter operator has multiple clients 
(i.e., users), each of whom submits jobs to the datacenter. 
Each job requires a certain amount of datacenter resources, 
e.g., CPU time, memory, and bandwidth; we suppose, as is 
often the case in practice, that these resources are not sub- 
stitutable. We index the resources with the variable i, and 
denote the amount of resource i required for each job by Rij, 
where j indexes the user type. Different types of users are 
distinguished by the different resource requirements of their 
jobs; in an extreme case, each user might submit jobs with 
slightly different resource requirements and thus belong to 
a distinct type. Multiple users could also be grouped to- 

^If the time intervals are very short, this pricing is sometimes 
called resource-as-a-service, i.e., the selling of individual re- 
sources at very short time granularities [J. 



gether into the same type, if their jobs have similar resource 
requirements that we can approximate as being the same. 
We fix the number of user types as n and the number of re- 
sources as m. If users are grouped together into a few types, 
then we could have n < m; otherwise, in general we have 
n > m as there are many more user clients than types of 
resources. 

When deciding how many jobs to submit, users take into 
account the prices set by the datacenter operator, which 
yield a cost per job for each user j. The structure of this 
pricing plan may vary; Section [3] introduces three different 
ways (bundled, resource, and differentiated pricing) in which 
the prices may be set. In all three pricing plans, each user 
has a fixed per-job cost, which we denote as rj for user j. 
This cost accounts for both the resource prices and the vol- 
ume discounts, which are taken to be separate functions of 
the amount of each resource required by the user. We pa- 
rameterize the volume discount with a constant 7 € (0, 1]; if 
a user requires RijXj amount of resource i, where Xj denotes 
the number of jobs submitted by user j, then the user is 
charged according to the discounted quantity _R7 a;J. Thus, 
a larger 7 corresponds to a smaller volume discount, and 
7 = 1 corresponds to no volume discount. Since the amount 
of resources required is an affine function of the user's de- 
mand, we let rj denote the per-job cost with the volume 
discount and separately account for the discounted term. 

Users derive utility from the number of jobs that the data- 
center processes, but lose utility from the amount that their 
jobs costQ We use xj to denote the number of jobs processed 
for each user j, and use the function Uj(xj) to denote the 
utility from these jobs. Since a user's utility increases with 
the number of jobs processed, we follow the economic prin- 
ciple of diminishing marginal utility and assume that Uj is 
a strictly increasing, concave function, expressed in units of 
dollars. Subtracting the cost rjx'j from the utility of pro- 
cessing Xj jobs, we find that each user j maximizes 

U,{x,)-r,x]. (1) 

For simplicity, we assume that Uj is continuously differen- 
tiable for Xj > 0. We suppose that the users have sufficient 
available cash to pay for any number of jobs Xj , so long as 
(m is positive; for instance, enterprise users would meet this 
requirement. Users thus have no explicit budget constraint; 
they submit eis many jobs as required to maximize ^ for a 
given set of prices. 

An example family of utility functions are the so-called 
isoelastic functions 



Cj \og{x) 



^^ a,€(0,l) 



(2) 



commonly used in economics p2]. In ((2)1, aj parameterizes 
the concavity of the utility function (i.e., quantifying the 
degree of diminishing marginal utility) and Cj > is a pos- 
itive constant that scales the utility level for user j. As aj 
increases, Uj becomes more concave, and the user becomes 
more price-sensitive since the cost term rjX*^ term becomes 
more dominant. 

We assume that the user receives zero utility if no jobs 
are processed; this assumption may be enforced by taking 

^We assume throughout that the operator sets prices so that 
all submitted jobs can be processed with the resources avail- 
able. "Submitted" and "processed" can thus be used inter- 
changeably. 



Uj{0) = 0. The maximizer x* of ([ij then satisfies 

Uj {x*j) = rj'yx*j''~\ (3) 

where x^, a function of the per-job price rj, denotes the 
utility-maximizing number of jobs. 

We assume that the operator chooses a value of 7 such that 
a solution to ^ exists, and that ([TJ has non-positive second 
derivative at this value of x* , ensuring that the solution is 
indeed a maximum. Mathematically, the second derivative 
condition may be expressed as 



U'/ {x*) < 7(7 - 1)= 



(4) 



For instance, the isoelastic utility functions satisfy this con- 
dition if 7 > l — ttj. We can then show that user j's demand 
Xj is decreasing in rj. By differentiating ((3)1, we find that 
this rate of decrease is 



yx 



,7-1 



[/; {x*) + 7(1 - 7)a 



(5) 



which is negative if (jl} holds. 



3. PRICING DATACENTER JOBS 

The presence of multiple datacenter resources allows the 
datacenter operator to offer many different pricing plans, 
each of which may be combined with volume discounts. In 
Section 13.11 we discuss the three pricing plans introduced 
in Section ri.2l bundled, resource, and differentiated pricing. 
We incorporate these prices in the user-optimization frame- 
work introduced in the previous section and then compare 
the optimal prices and resulting revenue in Section [3.21 

3.1 Pricing Strategies 

We first consider bundled pricing, in which the operator 
groups different resources in a fixed ratio-for instance, 1 
CPU and 2 GB of RAM might be grouped together as a 
bundled resource. The operator can then charge users a 
unit price per bundle of resources required for the user's 
jobs. We let bi denote the amount of each resource i = 
1, . . . ,m in each bundle, and let p denote the per-bundle 
price. Using the notation from Section [2] to describe users' 
utility functions and per-job resource requirements, we find 
that each user j requires 



^j = max 



Rdj 



(6) 



bundles in order to complete one job. To calculate the per- 
job cost, we incorporate the volume discount introduced in 
the previous section, parameterized by 7 £ (0, 1]. The per- 
job cost then becomes rj = t^]P- Moreover, the number 
of bundles available is min^ d/bi, where d denotes the ca- 
pacity of resource i. Thus, the operator faces the resource 
constraint 



max i 
^ i \ bi 



< mm — , 

i bi 



(7) 



where (mJ p) denotes the user's demand for jobs at price p, 
given the bundle {bi, . . . ,bm)- The resulting revenue under 
bundled pricing, pt, may be expressed as 



Pb 



(8) 



We next observe that p;, in (|8]) is a non-differentiable func- 
tion of the amount of each resource i (i.e., bi) in the bundle; 
thus, the operator will likely find it difficult to optimize over 
the bi. Instead, it can fix these bi and simply optimize over 
the unit price p. For instance, the bi might be chosen as 
proportional to the resource capacities d, i.e., such that 
for some constant b, b = Ci/bi for each resource i. In this 
case, the user's bundle requirement Q is proportional to 
her dominant share, which is defined as follows: 

Definition 1. A user's dominant share is the maximum 
share of any resource allocated to the user. Thus, if a user 
receives RijX* amount of each resource i, her dominant share 
would be msLXiRijXj/bi, i.e., the bundle requirement ([SJ 
multiplied by b. 

In Section r4.2l we show that users' dominant shares may be 
used to define the fairness of their utilities received. 

While bundled pricing is convenient for the operator in 
the sense that only one price variable needs to be chosen, 
bundled pricing may lead to wasted capacity. Since different 
users' jobs have different resource requirements, the ratio of 
resources in each bundle will not in general match the ratio 
of per-job resource requirements for a user. Thus, some users 
may be forced to purchase excess capacity of some resources- 
for instance, a user with a per-job requirement of 2 CPU 
cores and 1 GB of RAM would need to purchase 4 CPU cores 
and 1 GB of RAM for each job if the operator offered bundles 
of 4 CPU cores and 1 GB of RAM. To avoid this wasted 
capacity, which could potentially be sold to another user 
with different resource requirements, an operator could offer 
resource pricing, in which the unit price of each resource 
is independently determined by the operator, and users are 
permitted to purchase exactly as much of each resource as 
they need to process their jobs. 

In this pricing framework, we let pi denote the unit price 
of each resource i, so that each user j's per-job cost is rj = 
"127^1 P^^ij- Thus, the operator's total revenue pr is 

m n / / m \ \ ^ 

Pr = ^ ^ I R^jX* I p,R,j . (9) 

i = l i=l \ = l / / 

The resource constraints, which require that the operator be 
able to provide enough resources to each user, can then be 
expressed as 

n 

J2R^jXj<C^ (10) 

j = l 

for each resource i. The left-hand side corresponds to the 
sum of users' required amounts of resource i, while the right 
side is simply the capacity of resource i. 

We next note that resource pricing may be viewed as a 
form of bundled pricing in which the ratio of resources in 
each bundle exactly matches that required by users. This 
relationship between bundled and resource pricing suggests 
that the operator can offer a third, even more customized 
type of pricing-differentiated pricing, in which the oper- 
ator not only allows users to receive only as much of each 
resource as they need, but independently sets the prices of 
these customized bundles. Thus, the operator can set a dif- 
ferent per-bundle price pj for each user j. We suppose that 
one bundle exactly equals the user's resource requirements 
for one job, so that the per-job cost rj — Py The operator's 



revenue pd under differentiated pricing is then 

n 

and the resource constraints are exactly those under resource 
pricing (fTO)) . 

3.2 Comparing Pricing Plans 

In this section, we compare the operator's revenue under 
the three pricing plans introduced in the previous section. 
We suppose that for any of the three pricing plans, the op- 
erator chooses the prices so as to optimize a given objec- 
tive function of users' per-job costs and demands, e.g., the 
revenue H8I9I11|I . In Section [4.21 we define the fairness of 
users' utilities, which form another family of possible objec- 
tive functions. 

We first give sufficient conditions under which the objec- 
tive value under bundled pricing is at most that under re- 
source pricing: 

Lemma 1. If Rij/bi for each user j is maximized by re- 
source i = k, then the maximum objective function value 
with bundling does not exceed the maximum objective with 
unbundled resources. 

Proof. Suppose without loss of generality that k = 1. 
Each user's per-job bundle requirement is then R\j/b\. Let 
the optimal price with bundling be denoted by p^ . Then if 
the operator charges pi — p' jbx for resource 1 and noth- 
ing for any other resource, each user's per-job cost without 
bundling is Rijp'' /bi, which equals the cost with bundling. 
Each user's demand is solely determined by the price per 
job, so each user demands the same amount of jobs with and 
without bundling, and the resource constraints are satisfied. 
Since the objective function can be written entirely in terms 
of users' per-job costs, the value of the objective function 
is also unchanged. Thus, the maximum objective without 
bundling is at least as much as that with bundling. □ 

Given a set of resource requirements, a bundle always exists 
that satisfies the requirements of Lemma[T]-for instance, the 
operator might simply set 6i to be extremely small. Such 
a scenario, however, is somewhat artificial. If an operator 
instead sets the ratio of resources in each bundle equal to 
the ratio of resource capacities, then Lemma [1] holds if each 
user has the same dominant resource, defined as follows: 

Definition 2. A given user's dominant resource is the 
resource k which maximizes Rij jd over all resources i. 

We thus have the following corollary: 

Corollary 1. Suppose that the ratio of resources in each 
bundle is equal to the ratio of resource capacities. Then if 
each user has the same dominant resource, the operator's 
maximum objective function value with bundling does not 
exceed that without bundling. 

Proof. If the amount of each resource i in the bundle is 
bi, then there exists a constant B such that Bbi = d for 
each resource i. Thus, the resource k maximizing Rij jd 
also maximizes Rij /hi, and Lemma [1] applies. □ 

Intuitively, one might expect a result of this type, as the 
operator can choose only one price p under bundled pricing, 



while it chooses m > 1 resource prices pi under resource 
pricing. We note, however, that in general the operator's 
revenue under bundled pricing may not be less than that 
under resource pricing-under bundled pricing, users often 
are required to purchase excess capacity, and the resulting 
excess revenue may offset the operator's decreased flexibility 
in choosing prices. However, when comparing resource pric- 
ing and differentiated pricing, we obtain a stronger result: 

Proposition 1. // an operator prices jobs by unit prices 
Pi for each resource i, its maximum objective function value 
does not exceed the optimal value under differentiated pric- 
ing. If the matrix of resource requirements R has rank n, 
then the optimal objective function value under differentiated 
and resource pricing is the same. 

Proof. Suppose that the optimal prices under resource 
pricing are pi for each resource i. Let the differentiated 
prices p^ = Xlfci RljPi^ so that the price per job does not 
change for any user. Then Xj, the user's demand for jobs, 
does not change either, as it depends solely on the per-job 
cost. The objective function value under these differentiated 
prices thus attains the maximum revenue under resource 
pricing. Moreover, since the resource constraints depend 
only on the number of jobs xj, these differentiated prices 
are a feasible solution. 

If the matrix R has rank n, then the resource prices may 
be set in such a way that the per-job cost for each user equals 
that under differentiated pricing. Thus, since the objective 
depends only on per-job costs, the optimal objective value 
under resource and differentiated pricing is the same. □ 

In general, a datacenter operator will have several more 
clients than resources (i.e., m < n)-the number of data- 
center resources will generally be a small number, e.g., two 
for CPU cores and memory. However, if we instead group 
users into a few distinct types, then the m x n matrix of 
resource requirements R may have rank n < m. 

4. FAIRNESS AND REVENUE 

In this section, we examine a datacenter operator's two 
objectives: revenue and fairness. We flrst characterize the 
operator's revenue in Section|4?TJ and then consider fairness 
in Section [4.21 Section [4.31 then constructs an optimization 
framework that takes into account both revenue and fair- 
ness. We derive bounds on the tradeoff between fairness 
and revenue and give an algorithm that finds the optimal 
prices for bundled, resource, and differentiated pricing at 
various points on this tradeoff curve. 

4.1 Revenue and Resource Constraints 

From the previous section, we see that the operator's rev- 
enue may be expressed as 

fpEi=i (wa;* i^l]p)y 
P = { YZiPillU (^^^^i {TZiP^R^.))'' (12) 

for the bundled, resource, and differentiated pricing plans 
respectively. Should the operator wish to maximize its rev- 
enue, a natural approach would be to take the first derivative 
of the revenue with respect to each price variable and set it 
equal to zero. In this case, however, we can show that the 



revenue is always decreasing in each price variable, assuming 
that the utility function Uj is not too concave compared to 
the cost rjX*"^. Intuitively, each user j's total utility should 
grow slowly enough with the prices so that the optimal de- 
mand X* is well-defined, yet grows sublinearly so that the 
overall amount paid rjx*'' decreases with the prices \bfp. 

Proposition 2. Revenue p is decreasing in price if, for 
all users j , 

< U'/ {x*) + ^r.xf-^ < ^\,xY'-\ (13) 

where rj denotes the per-job cost for user j and is an affine 
function of the prices offered. For instance, the isoelastic 
utility functions satisfy these criteria if, as assumed in 
Section\^ 7 > 1 — aj . 

Proof. We take the first derivative of the revenue p with 
respect to a price variable pk (which may correspond to any 
of the three pricing plans) to obtain 

Since Xj' < 0, this derivative is negative if, for each user j, 

-X. > 

We can then use ([5]) to obtain the equivalent condition 

rr" ^ *7 — 2 

(7, > --yrjXj 

which is the first inequality in (|13|l . The second inequality 
follows from our assumption of concavity (|4]) in Section [2l 

If we assume isoelastic utility functions for each user, as 
in ([2]), then (|13p becomes 

< -CjajXj ' + ^rjXj < 7 rjX^ 

where x* — (rj/cj)^^^^"'^^ '\ Thus, we find the condition 

< —ctj-f 4- 7 < 7^, 

or equivalently < 1 — aj < 7, which we have assumed to 
hold in Section H □ 

Thus, the operator can maximize its revenue by offering 
the lowest possible prices compatible with its resource con- 
straints, i.e., such that the resources required to process all 
users' jobs do not exceed the resource capacities. We next 
show that if users have isoelastic utility functions then 
the amount of each resource required to process users' de- 
manded jobs is a decreasing function of the prices. The re- 
source constraints thus impose a lower bound on the prices 
offered; in fact, they define a convex set of the prices: 

Lemma 2. The set {p | RiX*(r(p)) < d, i = 1, 2, . . . , n} 
is convex, where r is a vector of the per-job price for each 
user and p a vector of the prices chosen by the operator. 
Moreover, RiX*(r(p)) is decreasing in each price pk- 

Proof. We consider only resource pricing; the result may 
be similarly proved for bundled or differentiated pricing. It is 
sufficient to show that Xj is a convex function of the prices 
p for each user j; since RiX* is an affine combination of 
convex functions, the lemma follows. We flrst differentiate 
and flnd that 

dpk 1 - Qj - 7 J 



since 7 > 1 — Qj by assumption. Upon differentiating a 
second time, we find that 



dpkdpi (1 - Qj - 7) 



Thus, the second-derivative matrix of x* equals RQR 
where Q is a diagonal matrix with entries 



Qn=1 



+7 



(1 - a, - 7) 



2aj+27-l 



A sufficient condition for the second-derivative matrix to be 
positive semi-definite is then that 

which holds for each user j. □ 

We have thus shown that in order to optimize revenue, the 
operator should offer the lowest prices possible. However, 
in doing so the operator could inadvertently favor one user 
over another, possibly introducing user dissatisfaction. For 
instance, suppose that one user k requires a relatively large 
amount of one resource I (e.g., this user runs extremely 
computationally-intensive jobs and requires a large number 
of CPU cores). Under resource pricing, the operator might 
then increase the price pi of this resource slightly, in order to 
offer much lower prices for the other resources while keeping 
the total demand for resource X]j=i Rij^^ (X^I^i RijPi)^ 
beneath the capacity C;. But in this scenario, user k would 
then have a higher per-job cost X^I^i RikPi than other users, 
and thus process a lower number of jobs and receive lower 
utility Uj{Xj) — rjXj. In the next section, we explore ways 
to limit inequality among users by measuring the fairness 
resulting from a set of offered prices. 

4.2 Fairness 

When users submit jobs to the datacenter operator, they 
are allocated a given amount of each resource, which suffices 
for the job to be completed. Thus, the operator is perform- 
ing a multi-resource allocation among users, as considered in 
[51 [TT]. In these works, the fairness of a resource allocation 
is measured in terms of the number of jobs processed for 
each user-users are assumed to receive utility not from the 
resources allocated, but rather from the jobs that those re- 
sources allowed them to complete. In our framework, since 
users use utility functions to measure their utility from pro- 
cessed jobs, we measure the fairness of the utility value re- 
ceived (i.e., Uj = Uj [x*) — rjx*'^) by each user j, rather 
than simply measuring the number of jobs processed for dif- 
ferent users. 

When measuring the fairness of a distribution, we con- 
sider two factors: equitability and efficiency. Equitability 
refers to the idea that users should receive equitable utility 
values, e.g., instead of assigning prices so that one user pro- 
cesses a much larger number of jobs and receives a higher 
utility than another user. However, in some cases one can 
increase a user's utility level without significantly diminish- 
ing other users' utilities: a more unequal allocation can re- 
sult in more efficiency, or a larger amount of total utility 



across users. Following [TT]'s exploration of this equitability- 
efficiency tradeoff for multi-resource allocations, we utilize 
the following fairness functions: 



sgn(l-/3) K]C/ 
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(14) 



where /3 7^ 1 and A respectively parameterize the equitabil- 
ity and efficiency [T^Q The term 1 ^ function 
simply of the total utility across users, may be regarded 
as the efficiency component of this fairness function; as |A| 
increases, the total utility is more heavily weighted in the 
function. The remainder of the function, parameterized by 
a factor /3, measures the equitability of the utility distribu- 
tion. This component is homogeneous in the total utility 
value, and thus measures only the relative values of the util- 
ities received by each user. We note that if A = /?~^ — 1, 
then maximizing (|14|) is equivalent to maximizing 
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where /3 > parameterizes the type of fairness under con- 
sideration. This fairness function corresponds to the well- 
known a-fairness function with a = /3; we thus observe that 
as P grows, (|15|) becomes "more fair" [12| . 

We next show that when users have isoelastic utility func- 
tions, the fairness function H15[l becomes a weighted fairness 
function of the number of jobs processed for each user. To 
do so, we first note the following lemma: 

Lemma 3. Given a set of prices p, then assuming users 
choose their demand so as to maximize their isoelastic util- 
ities {3), 
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(16) 



Thus, the utility received from each user equals the amount 
paid rjX*^ , multiplied by a price-independent constant. 

By choosing the prices to offer, the operator thus determines 
both the number of jobs processed x* as well as the coef- 
ficient (7/(1 — Qj) — 1) rj of x*'' in users' total utility. If 
7 = 1, this coefficient simply multiplies a-*, or the number 
of jobs processed by user j. 

The fairness functions H14[) and H15|) assume that all users 
are equal, in the sense that the utility received by each 
user is treated the same. However, in practice users are 
heterogeneous-they each submit a distinct type of jobs to 
the datacenter operator, with different resource requirements 
and different utility functions. Thus, it may not be equitable 
to treat all users equally. In Lemma [3l we account for users' 
differing utility functions with the constant 7/(1 — aj) — 1. 
This constant weight is decreasing in oij ; thus, as a user be- 
comes less price-sensitive (aj decreases), the weight in the 
fairness function increases. More price-sensitive users tend 



■^These functions are the uniqu e fa irness functions satisfying 
four desirable fairness axioms [T^]. In Appendix [UJ we con- 
sider two additional properties (envy-freeness and Pareto- 
efficiency) that are not axiomatically satisfied. 



to receive lower utility values for the same number of jobs 
due to lower values of aj in their utility functions an 
effect that is offset with this weight. 

By examining the value of the per-job cost rj for bun- 
dled, resource, and differentiated pricing, we can gain fur- 
ther insight into the interpretation of fairness for each of 
these pricing plans. Under bundled pricing, we find that 
rj — jijp. Supposing that the ratio of resources in each 
bundle equals the ratio of resource capacities, we see from 
((51) that fJ,jXj is simply user j's dominant share; thus, if 
there is no volume discount (7 = 1), then the fairness weight 
(7/(1 — aj) — 1) rj is simply the price p multiplied by 1/(1 — 
aj) — l and the user's dominant share. Thus, the fairness on 
users' utility values can be viewed as a (weighted) fairness on 
the amount of resources required by each user. Users whose 
jobs have larger resource requirements (larger fij) may thus 
process fewer jobs-their jobs require more of the limited re- 
sources, thus detracting from other users' jobs. This fairness 
on dominant shares metric has been previously proposed for 
general allocations without pricing [5] [TT] . 

If we instead consider differentiated pricing, we note that 
the weight (7/(1 - Oj) - 1) rj = (7/(1 - Oj) - l)p^ in ^ 
is completely determined by the operator; users' resource 
requirements only enter into the choice of p^ through the 
demand functions Xj . This pricing scheme thus corresponds 
to a fairness function that does not as explicitly account for 
resource requirements. 

Under resource pricing with no volume discount, we ob- 
tain a fairness metric that lies between the fairness with 
bundled and differentiated pricing. With resource pricing, 
we see from (|16p that each Xj^s weight in the fairness func- 
tion (fn]) becomes (7/(1 — cej) — l)X]™iPiflii- Thus, the 
number of jobs is multiplied by a constant depending on aj, 
multiplied by a weighted sum of a user's resource require- 
ments. Users with higher resource requirements will thus 
process fewer jobs. However, unlike bundled pricing's domi- 
nant shares, the exact weight for each resource is determined 
by its price, which may be seen as a proxy for how "valu- 
able" different resources are. More valuable resources will 
have higher prices and thus yield higher weights. 

4.3 Fairness-Revenue Tradeoffs 

A datacenter operator wishes to optimize both revenue 
and fairness. In this section, we consider a weighted sum of 
the revenue (|12[) introduced in Section 14.11 and the fairness 
(|15[l introduced in Section [4.21 

up{p) + Fp (p) (17) 

where > is the weight parameter determining the opera- 
tor's relative emphasis on fairness or revenue. By optimizing 
a weighted sum of revenue and fairness, the operator can op- 
timize both its revenue and the fairness of users' resulting 
utility distribution. We consider users with isoelastic utility 
functions ((2]) and show that, when the revenue weight v is 
sufficiently small, this optimization problem becomes con- 
vex. While this result indicates that an operator can more 
easily optimize fairness, possibly leading to a loss in rev- 
enue, we can adapt the proof of Lemma to show that this 
tradeoff is limited. 

We first derive conditions on the weight u under which 
maximizing (|17p subject to the resource constraints is a con- 
vex optimization problem: 



Data: Parameters Oj , u. and 7, tolerance e > 0, update 

parameter /i > 1. 
Result: Optimized resource prices. 
Initialize i 1; 
while t < m/tol do 

Find Pk(t) satisfying /t(p) = 

-t.vp(p) - tvf,(p) + Er=i ^.^^.\ll^[Ti)-c. = ° 

where k indexes the number of solutions; 
Choose k such that Pk(t) minimizes ft{p) — 
-typ{p)-tVF^{p)-Y.T=i^o& (-Rix* (E[^iPiR.ij) +Ci); 
Update p Pk(t); 
Update t <— fit; 

end 

Algorithm 1: Interior-point optimization of (|17p . 

Proposition 3. Suppose that /3 > 1 and that 
< ^ < {^^^) 7^^"' im " - 7) 

X max — — 

for all users j. Then {_??[ ) is a concave function of the prices 
and, by Lemma\^ maximizing j j7[ j subject to the resource 
constraints is a convex optimization problem. 

Proof. See Appendix [B] □ 

If u satisfies Prop. Os conditions, standard convex opti- 
mization solvers may be employed to solve for the optimal 
prices. Even when u is larger than the bound given, however, 
we can adapt these algorithms to find the optimal prices, as 
shown in Algorithm [1] for the interior-point algorithm. Intu- 
itively, the interior point algorithm starts from a set of prices 
p and then uses Lagrange multipliers on the problem con- 
straints to iterate the values of p so as to increase the objec- 
tive while remaining within the problem constraints. In our 
case, differentiating Fp for /3 > 1 shows that the objective 
P7[l is decreasing in each price variable-thus, intuitively, the 
operator searches for the lowest feasible prices. Using this 
intuition, one can adapt standard proofs of convergence for 
the interior-point algorithm to show the following: 

Proposition 4. Algorithm\^ converges to a price vector 
p maximizing the objective | [j7| ). 

In the case of only one resource, the lowest feasible price for 
this resource is the price for which the resource constraint 
is tight; by Lemma [2] since the demands x* are decreasing 
in the price variables, a unique price exists satisfying this 
property. Thus, we obtain the following corollary: 

Corollary 2. Suppose that the operator offers a bundled 
pricing plan. Then its objective \17^ for any revenue weight 
V is maximized at the lowest feasible price, which is the zero 
of the bundled resource constraint "^Z^^iPl^J^^iP)^ ~ 1- 

We finally note that Algorithm [T] holds only for fixed vol- 
ume discounts 7. However, if the operator's optimization 
problem is convex, as in Prop. [S] then the optimal prices 
for each 7 can be determined efficiently. The operator can 
then perform a line search to determine the optimal volume 
discount. Section [5. 31 numer icallv characterizes the achieved 
fairness and revenue for different volume discounts. 

The upper bound on the revenue weight 1/ in Prop, praises 
some concern that the operator may overly weight fairness 



and consequently lose revenue. However, when users have 
isoelastic utility functions, we can in fact bound the achieved 
revenue in terms of the achieved fairness when the fairness 
parameter /? > 1; thus, even if the fairness term in (|17|) 
dominates the revenue term, the operator's revenue is still 
lower-bounded away from zero. Moreover, if /3 < 1, a class of 
fairness measures not included in Prop. [3l then the achieved 
fairness can be lower-bounded in terms of the achieved rev- 
enue. Thus, emphasizing fairness in the operator's objective 
need not lead to a significant loss of either revenue or of the 
fairness measured by metrics besides those in Prop. [S] 

Proposition 5. Suppose that /3 > 1. The revenue p (p) 
can then be lower-bounded in terms of the achieved fairness 



P(p)> (F/3(p)(l-/3))^E 
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On the other hand, if P < 1, then the fairness value Fi3{p) 
may be lower-bounded in terms of the achieved revenue: 



1-13 



Oik 



axj Uj . Intuitively, a lower value of P more empha- 
sizes efficiency: A=l//3 — 1 in \14^ is increasing in p. By 
Lemma\^ the efficiency, or total utility received, is simply 
a scalar multiple of the revenue. A higher /3, on the other 
hand, gives more emphasis on fairness, and thus cannot be 
lower-bounded by the revenue. 



Proof. See Appendix[ 
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5. NUMERICAL ILLUSTRATIONS 

In this section, we use a six-hour workload trace from a 
Google cluster to illustrate the effects of resource capacity 
and volume discounts on a datacenter operator's achieved 
revenue and fairness for different pricing plans [9]. In par- 
ticular, we find that increasing resource capacity allows im- 
provements in both fairness and revenue, but that these re- 
sults are highly dependent on the heterogeneity of the users' 
resource requirements. Varying the volume discount offered 
allows an increase in fairness, but results in decreased rev- 
enue and more leftover resources under resource and differ- 
entiated pricing. We find that for a range of resource ca- 
pacities and volume discounts, all three pricing plans yield 
comparable revenues, but differentiated pricing permits a 
significantly higher fairness. 

Our workload trace includes 9218 jobs divided into 176580 
tasks; each task runs on a single machine within the cluster. 
The amount of memory (RAM) and fractional number of 
CPU cores taken up by each active task was recorded at five- 
minute intervals over 6 hours; these measurements were then 
passed through a linear transformation to ensure anonymity. 
For this reason, we omit units in our discussion here. We add 
each job's resource usage over all the time intervals recorded 
to find an average per-job CPU usage of 0.136 and memory 
usage of 0.182. 

The jobs in the Google trace show a large variation in 
resource usage; the CPU and memory usage distributions 
have respective standard deviations of 13.4 and 18 times 
their means. To simplify our simulations, we exclude jobs 
whose total usage of either CPU or memory lies more than 
one standard deviation away from the mean. We then use 
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Figure 1: Jobs in the Google trace, clustered based on the 
total CPU cores and memory used in a 6-hour interval. 

fc-means clustering to group jobs into three different clusters 
or types by their CPU and memory usage. Figure [1] shows 
the resulting clusters of jobs; we ran 30 instances of the k- 
means algorithm and chose the result minimizing the intra- 
cluster distance. We take the fc-means centroid points to 
be the resource requirements of each job type: type 1 jobs 
require 0.4 CPUs and 2.7 units of memory, while type 2 jobs 
require only 0.01 units of CPU and 0.02 units of memory 
and type 3 jobs require 0.6 units of CPU and 0.5 units of 
memory. We associate a user type with each type of job and 
assume that these users have isoelastic utility functions ([2]), 
with parameters Cj = 1 and Oj = 0.4, 0.7, 0.5 for j = 1, 2, 3 
respectively. Taking the capacity of each resource to be 6 
units, the operator's objective (|17|l then becomes: 



max iyp{p) -\- Fa(p) 



(18) 



s.t. {0.4a;^ -f 0.01^:2 + 0.6x3, 2.7x1 -h 0.02a;2 -\- 0.6x3} < 6, 

where p is the vector of prices offered for either bundled, re- 
source, or differentiated pricing. For scalability, we suppose 
that there are 10 users, with 1 user each of types 1 and 3 
and 8 users of type 2. 

We first consider the effects of resource capacity on the op- 
erator's revenue and fairness in Section [5.11 and then show 
that these tradeoffs can change dramatically with the distri- 
bution of users (Section l5.2|) . Section [5]3] then considers the 
effects of volume discounts. Throughout our discussion, we 
take the fairness parameter /3 in (I18|l to be relatively large, 
at /3 = 20; this choice of /? allows us to approximate max-min 
fairness, i.e., maximizing the minimum utility value. Larger 
values of /3 thus impose a stricter fairness requirement than 
lower ones, and have been used previously as fairness bench- 
marks in multi-resource allocation |6l [11] . 



5.1 Resource Capacity 

We first examine the effect of the available capacity on 
the operator's achieved revenue and fairness. We vary the 
memory capacity from 1/3 to 8 and use the interior-point 
algorithm (c/. Algorithm [l]) to solve for the optimal values 
of the bundled, resource, and differentiated prices for differ- 
ent revenue weights v in (|18|l . Figure [2] shows the achieved 
revenue and fairness for each pricing plan. As the memory 
capacity increases in Fig. [2] the operator's fairness and effi- 
ciency under each pricing plan both improve: the operator 
has more memory to allocate, and thus more flexibility in 
setting the prices. Mathematically, we see that one con- 
straint in the optimization problem (|18|l has been relaxed, 
allowing the operator to increase the value of its objective 
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(a) Bundled pricing. (b) Resource pricing. (c) Differentiated pricing. 

Figure 2: Achieved fairness and revenue for a range of revenue weights v in (|18|) and memory capacities. On each contour 
(individual contours only visible for differentiated pricing), v is fixed and the capacities are varied. For differentiated pricing, 
the contours reveal a range of tradeoffs for each fixed capacity and varied weight v. 




Figure 3; Utility received by the three user types as the RAM (memory) capacity varies for maximum fairness or revenue. 



function. We note that, as expected from Corollary O the 
achieved revenue and fairness do not change with v for bun- 
dled pricing. The optimal bundled price, which maximize 
both revenue and fairness, is simply the lowest price consis- 
tent with the operator's resource constraint. 

We next compare resource (Fig. I2b|) and differentiated 
(Fig. [2c)) pricing. Though differentiated pricing adds only 
a single additional price variable, it results in a significantly 
larger fairness for the operator, especially at low memory 
capacities. Moreover, varying the revenue weight v results 
in a much richer set of tradeoffs between fairness and rev- 
enue for differentiated, rather than resource, pricing. This 
result may reflect the large heterogeneity in users' resource 
requirements: while both type 1 and type 2 users require 
a very large amount of memory relative to their CPU re- 
quirements, type 2 users require significantly less memory 
and fewer CPUs than type 1 users. If the operator offers 
resource pricing, type 1 users are then very affected by the 
memory price, and their utility will be adversely impacted 
by an increase in this price. In fact, type I and type 3 users 
also require more CPUs per job than type 2 users, so their 
utility levels will also be adversely impacted by increases in 
the CPU price. At the same time, type 2 users will process 
many jobs due to their jobs' relatively low resource require- 
ments cost, allowing the operator to extract more revenue 
from type 2 users. Under differentiated pricing, the opera- 
tor can manipulate the prices of individual users to remove 
these constraints imposed by resource heterogeneity, allow- 
ing the operator to increase the fairness across users without 
a large decreases in revenue. 

We can validate the above explanation of the operator's 



low achieved fairness for bundled and resource pricing by 
calculating the utility received by each user under the three 
pricing plans. Figure [3] shows the utility values for each 
user when only fairness is optimized (i.e., = in HI}), 
and when only revenue is optimized (y — > oo). We see that 
type 2 users receive more utility than users of types 1 or 3; 
since type 2 users have far smaller resource requirements, 
this result is unsurprising. Users 1 and 3, however, receive 
almost the same utility under bundled and resource pricing, 
which is much less than that under differentiated pricing. 
As the capacity increases, however, their utility levels un- 
der resource pricing become visibly larger than those under 
bundled pricing, corroborating Fig. [5Is higher fairness val- 
ues under resource, rather than bundled, pricing. 

5.2 User Heterogeneity 

We next examine whether the dramatic changes in fairness 
for different pricing plans is still observed when users are less 
heterogeneous. We vary the fraction of type 1 and type 2 
users varies (we fix the fraction of type 3 users to be 10%) 
and fix the memory capacity to be 6 units. Figure 2] shows 
the resulting fairness and revenue tradeoffs; we see that a 
visible range of tradeoffs exists for both resource and differ- 
entiated pricing, and that the fairness for different revenue 
weights V is much less negative than in Fig. (2] As the frac- 
tion of type 1 users increases, the revenue decreases under all 
pricing plans: type 1 users require more resources than type 
2 users, so the operator receives more revenue from using 
its available resources to process type 2 users' jobs. Under 
bundled pricing, the fairness also increases with the fraction 
of type 1 users; all type 1 users receive the same (lower than 
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(a) Bundled pricing. 



(b) Resource pricing. 



(c) Differentiated pricing. 



Figure 4: Achieved fairness and revenue for a range of revenue weights v and distribution of user types. On each contour 
(individual contours not visible in (a)), the weight v is fixed and the fraction of type 1 users varies. For (b) resource and (c) 
differentiated pricing, the contours reveal a range of tradeoffs for each distribution of user types and varied revenue weights v. 



type 2) utility values, increasing equitability. Under resource 
or differentiated pricing, however, increasing the fraction of 
type 1 users decreases the fairness. To achieve a higher rev- 
enue, the operator processes more jobs from type 2 users, 
even when more type 1 users are present. 

As expected from Corollary (2] varying the revenue weight 
V has no effect on the utility or revenue from bundled pric- 
ing. Under resource or differentiated pricing, however, as the 
revenue weight v increases in Fig. |4l the operator's revenue 
increases for any given fraction of type 1 users. However, 
this increase in revenue results in a decrease in fairness: in 
particular, when a low fraction of users is present in resource 
pricing (e.g., the leftmost contour in Fig. I4bp . the revenue 
barely increases, while the fairness drops steeply as v in- 
creases. We can explain this result by examining the utility 
received by each user for different fractions of type 1 users, 
as shown in Fig. [S] 

We see from Fig. [5] that as the fraction of type 1 users 
increases, the minimum utility over all three users also de- 
creases, though type 2 and type 3 users actually receive more 
utility under bundled and resource pricing when the revenue 
is maximized. This is offset, however, by a decrease in utility 
for type 1 users, which results in a decrease in the minimum 
utility across users. This result likely comes from the fact 
that type 1 users require more resources to process their 
increased number of jobs, as the fraction of type 1 users in- 
creases. For bundled pricing, however, the minimum utility 
increases when either revenue or fairness is maximized, likely 
because type 1 users, who had previously received lower util- 
ities than type 2 or 3 users, could now process more jobs. 
When fewer type 1 users are present, the price of type 1 
users' jobs under resource or differentiated pricing may be 
set so that these users receive slightly higher utility values; 
since there are relatively few type 1 users, they do not take 
away many resources from the other types of users. 

5.3 Volume Discounts 

We conclude by examining a parameter set by the oper- 
ator: the volume discount 7. Figure |6] shows the fairness- 
revenue tradeoffs for a range of volume discounts 7. 

We see that for both bundled and resource pricing, vary- 
ing V in the operator's objective function (|18|l does not yield 
an appreciable tradeoff (for bundled pricing, this result is 
again Corollary [SJ . Moreover, as 7 decreases, the fairness 
decreases quite rapidly under both these pricing plans; once 



the revenue exceeds 50, the fairness value decreases exponen- 
tially. Differentiated pricing allows a wider range of much 
higher fairness values. As the revenue weight v increases 
from 0, the revenue under differentiated pricing increases 
rapidly, while the range of achieved fairness stays roughly 
the same; though each contour represents a linear increase 
in V, the resulting increase in revenue, measured by the "dis- 
tance" between the contours, is very pronounced. As v in- 
creases further, the contours grow closer together, but the 
range of achieved fairness decreases more rapidly. 

For larger revenue weights v, we find that the operator's 
revenue decreases with 7 for all three pricing plans, as a re- 
sult of decreased demand from users. This demand decrease, 
however, affects users in different ways and acts to increase 
fairness. When v = ^ under differentiated pricing, the rev- 
enue does not decrease monotonically with 7; for a small 
volume discount (7 near 1), the revenue initially increases 
and then decreases with 7. Unlike our results for varied 
memory capacities or distributions of users (Figs. [2]and|4]), 
the differentiated pricing contours for different v actually 
overlap: the operator can achieve the same combination of 
fairness and revenue by optimizing two different weighted 
sums of fairness and revenue while offering different volume 
discounts. 

We can explain the very negative fairness values observed 
in Figs. |6a] and I6bl for bundled and resource pricing by 
examining the equitability and efficiency components of the 
fairness function (|f 5p . We see that the equitability is quite 
negative, even for differentiated pricing, where the fairness 
values are relatively moderate (Fig. l6c)) . Thus, the very low 
fairness values observed in Fig. [6]may be attributed more to 
lack of equitability in the utilities received by different users 
rather than a low total utility. 

The relatively low efficiency values observed in Fig. [7] 
lead us to consider another measure of efficiency: leftover 
resource capacity. Though leftover capacity is not explic- 
itly accounted for in the user's optimization problem (|18p - 
capacity only enters through the resource constraints-the 
leftover capacity may indicate the potential additional rev- 
enue that an operator could achieve, e.g,. if users' demand 
functions change due to different volume discounts offered. 
We find that for all volume discounts 7 and weights v, only 
CPU cores are leftover; as the revenue increases, the leftover 
capacity increases for both resource and differentiated pric- 
ing. Thus, a volume discount reduces the "wasted" capacity 




0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 

Fraction of Type 1 Users Fraction of Type 1 Users Fraction of Type 1 Users 
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Figure 5: Utility received by the three user types as the fraction of type 1 and 2 users varies, maximum fairness or revenue. 
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(a) Bundled pricing. 
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Figure 6: Achieved fairness and revenue for a range of revenue weights v and volume discounts 7. On each contour (individual 
contours only visible for differentiated pricing), v is fixed and 7 varies. For (a) bundled and (b) resource pricing, the optimal 
fairness and revenue values do not change significantly with v. For (c) differentiated pricing, the contours reveal a range of 
tradeoffs for each fixed 7 and varied v. 




Figure 7: Equitability and efficiency components of the fair- 
ness function (|f 5[) as the revenue weight v varies in the op- 
erator's objective function (|f 8|) . 
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Figure 8: Leftover CPU capacity versus achieved revenue 
as the volume discount 7 and revenue weight v vary in the 
operator's objective function (jf Sp . 



by increasing user demand and operator revenue. For bun- 
dled pricing, however, we have the opposite scenario: as the 
revenue and overall user demand increase, the leftover capac- 
ity also increases. With bundled pricing, users are forced to 
purchase excess capacity according to the ratio of resources 
in the resource bundle. As the volume discount increases, 
user demand goes up and users purchase more excess capac- 
ity, which is included in the leftover capacity shown in Fig. 
[7] This result thus highlights the "inefficiency" inherent in 
forcing users to pay for resources they do not need. 

6. CONCLUSION AND FUTURE WORK 

In this work, we develop a mathematical framework for 
evaluating three cloud pricing strategies-bundled, resource, 



and differentiated pricing-in terms of their resulting fair- 
ness and revenue. We first characterize user demand for 
resources as a function of the prices offered under these dif- 
ferent pricing plans, and then find the resulting revenue. We 
next formulate a family of metrics to measure the fairness 
of the offered prices and analyze the tradeoff between fair- 
ness and revenue. After showing some analytical bounds on 
this tradeoff, we finally use data taken from a Google cluster 
to numerically evaluate the impact of resource capacity and 
volume discounts on the operator's fairness-revenue trade- 
off. We find that differentiated pricing, even if it only adds 
one additional price variable compared to resource pricing, 
can significantly increase the fairness under bundled or re- 
source pricing, though revenue remains comparable across 



all pricing strategies. The operator can also increase both 
fairness and revenue through an increase in resource capac- 
ity, but the magnitude of this benefit depends heavily on 
the heterogeneity of users' resource requirements. Finally, a 
lower volume discount can be used to increase fairness, but 
at a loss of some revenue and increase in unused resource 
capacity. 

Throughout this work, we consider fairness and revenue 
solely in terms of prices. A real implementation, however, 
would allow us to explore a different aspect of fairness: user 
priorities. Instead of choosing prices so as to balance fair- 
ness and revenue, an operator could instead leverage the 
temporal dimension of job processing in datacenters to or- 
thognalize an operator's desire for fairness with its incen- 
tive to increase revenue. Users receiving lower utility values 
could be compensated with higher priorities, allowing their 
jobs to be completed faster. By implementing such a pri- 
oritized allocation scheme and devising algorithms for com- 
puting these priorities, we can evaluate whether setting user 
priorities, choosing job prices under one of many possible 
pricing plans, or a combination of both is most effective at 
balancing a datacenter operator's competing objectives of 
maximizing revenue and fairness. 
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APPENDIX 

A. INTRODUCING DEADLINES 

In this section, we show that our formulation in Sections[2] 
-3] may be altered to allow changes in user utilities and job 
demands over multiple time intervals. In formulating this 
problem, we account for varying job deadlines: users specify 
a completion deadline when a job is submitted, and the op- 
erator takes these into account when setting prices for each 
time interval. We suppose that the operator sets different 
prices in each time interval; then if the price optimization 
for the single interval is convex (e.g., as in Prop. [H]), so is 
the optimization over prices in all time intervals. We derive 
this extension only for resource pricing, but similar methods 
can be used for bundled or differentiated pricing. 

We suppose that when users submit jobs to the datacen- 
ter operator in each time interval, they specify the maxi- 
mum time to complete these jobs as well as the job resource 
requirements that, if met, will ensure that the job is com- 
pleted before the deadline. We suppose that it does not 
matter when a user receives the requested resources, so long 
as the resources are allocated in the requested ratio. For 
instance, a job with a deadline of three time intervals and 
a requirement of 6 CPU slots and 3 GB of RAM might be 
allocated 2 slots and 1 GB in each time interval; or 2 slots 
and 1 GB in the first, no resources in the second, and 4 slots 
and 2 GB in the third time interval; either allocation allows 
the job to complete before the deadline. In each time in- 
terval, each user submits one type of job to the datacenter 
(i.e., Rij is the same for all jobs of user j and all resources i 
submitted during a given time interval). If a user has mul- 
tiple types of jobs to run, these may be treated as coming 
from distinct users. One could also group users with similar 
per-job resource requirements. 

We consider a finite time horizon of T time intervals, in- 
dexed by s = 1, 2, . . . , T, and assume that all job deadlines 
fall before the final time T. At the beginning of each time 
interval s, the operator sets the resource prices pi^s for each 
resource i; denoting the vector of resource prices in period 
s by Ps, each user j then submits 2;* s(ps) number of jobs 
to the operator, with associated deadline r,,s. These jobs 
are charged according to the price pa, though the resource 
prices may change before the job completes. Users choose 
the number of jobs so as to ma^cimize their utility in period 
s, and do not account for any jobs that have been already 
submitted or may be submitted in future periods. For sim- 
plicity, we suppose that for each user j, all of user j's jobs 
submitted in a given period s have the same deadline. 

We now introduce the variables Xj^a(t), s < t, to denote 
the number of jobs submitted by user j in period s that are 
processed during time interval t. To ensure that all deadlines 
are met, we introduce the constraints 



(19) 



for each user j and time interval s; since a;* ^ is convex in 
the prices ps by Lemma [2] this constraint defines a convex 
set. The resource constraints for each period t then become 



^5;^i?.,(s)x,,,(t) <a. 



(20) 



for jobs submitted by user j in the time interval s. We note 
that these constraints are affine in the variables Xj^s(t)\ thus, 
the constraints (|19p and (|20p together define a convex set in 
the variables ps and Xj^sit). We then find that the revenue 
with deadlines may be expressed as 



s = l \j = l i = l 



(21) 



The fairness may be thought of as the sum of the fairness 
of the utilities received in each time interval; the fairness 
function is thus only indirectly dependent on the deadlines 
rj,s. Users specify the deadlines independent of the prices 
offered, so these should not affect their perceived utility or 
fairness. Thus, (|15p yields the fairness function 



1 ^ / " _ 
T3« E E (4 ,(p.), Ps 



(22) 

s = i = i / 

We note that since both the revenue \21\ and fairness (|22|) 
are simply the revenue and fairness in each time interval 
s added together, we may apply Prop. [3] to formulate a 
weighted sum (with weights possibly depending on the time 
interval) of both the revenue and fairness that is a concave 
function. We then have the following result: 

Proposition 6. The optimization problem 



(23) 



max '^v{s)\^xls''^Rij(s)p^,, 



+ 
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E^^J (a^j,s(Ps),Ps 
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EE-^'J^^^^J'''^*) 1 ^ * ^ l<i<m 

(24) 

J2^jAt) > xlsiPs); 1 < s < T; 1 < i < n 

t — s 

(25) 

is a convex optimization if, for each time interval s, the 
weight !/(s) is chosen so as to satisfy Prop. [5|/or the revenue 
and fairness in period s. 

B. PROOFS 
B.l Proposition |3] 

Proof. As in the proof of Lemma [21 we show the result 
for resource pricing; the proofs for bundled or differentiated 
pricing are analogous. We first note that the second deriva- 
tive matrix of (|17p may be written as 

i.V^p + V^F^= [R7j]Q[R7j]^, 
where Q is a diagonal matrix with entries 

27 + Qj-l 



1 — Ui 



(1 - aj - 7) 



+ 7^- 



7 + 



1 — a, 



1 - Qj - 7 



=1 j= 



(-l-g)(l-nj) + 27 



where Rij{s) denotes the per-job requirement of resource i 



and I^R-ijj is an m x ?i matrix with entry A 

sufficient condition for this second-derivative matrix to be 
negative-semidefinite is that Qjj < for eacli j. Thus, we 
find the condition 



u < 



. We can remove the price variables from this bound by 
observing that the resource constraints Rx* < C imply that 



for each resource i and user j; thus, 



a --(--y_l 

r . ^ > max l 

• V Hi 



We therefore find the sufficient condition 



u < 



1 — Ctj 



X max 



.a(Qj-i) ' 



(/?(!- a,) -7) 



Ri 



□ 



B.2 Proposition m 



Proof. Suppose that /3 < 1 in ([TSj. We first introduce 
the notation 

_ 7 + «j - 1 _ 7 -I 



and find that the fairness may be written as 

n 



1-0^ ' \cj ' 



l-B / n / s 2 1- 

P(P)^-V 7 ^^-^ 
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1-/3 Vl-«fc 

The first inequality uses the sub-additivity of the function 
f{x) = x^~^ , and we let k be such that = miuj Oj. 
We now suppose that /3 > 1 and find that for each user j, 

-,{l-fl) (l-Qj)(l-/3) 

which is equivalent to 

-7 1 — a ■ 

'^i J 

Adding these lower bounds and using the definition of Uj, 
we find that 

n 

P(P) > {Fp{p){l - /?)) ^ E ' 



^ 7 + Oj - 1 



C. FAIRNESS PROPERTIES 

While the fairness functions (|14l) are the unique functions 
satisfying certain desirable fairness axioms, other desirable 
fairness properties are not axiomatically satisfied by these 
functions. In this appendix, we consider two such properties: 
envy-freeness and Pareto-efficnecy. 

Definition 3. Envy-freeness holds if and only if no user 
envies another user's allocation. Mathematically, let Tij de- 
note the amount of resource i received by user j. User j can 
then process maxiVij/Rij number of jobs. Envy-freeness is 
defined as the property that maxirij/Rij > maxiVik/Rij 
for any j 7^ k. In words, no other user's allocation would 
enable a user to process more jobs than her allocation would. 

Definition 4- A function / is Pareto-efFicient if, when- 
ever X Pareto-dominates y (i.e., Xi > yi for each index i and 
Xj > Vj for some j), /(x) > /(y). 

We first show that envy-freeness always holds for an allo- 
cation of utilities if users optimize the number of jobs that 
they submit under bundled or resource pricing: 

Proposition 7. Suppose that each user chooses a num- 
ber of jobs to submit, Xj{rj (p)), so as to maximize her util- 
ity function. Then for any given set of resource or bundled 
prices p, if two users switch their corresponding resource al- 
locations and are required to pay the difference in resources 
received, neither gains any utility. If users do not need to 
pay this difference, then a user may gain utility by switching 
allocations. 

Proof. The first part of the proposition is trivial, as for a 
given set of prices, users are assumed to optimize the num- 
ber of jobs processed. Since all users are charged at the 
same per-resource or per-bundle rates, if switching resource 
allocations alters the number of jobs processed for either, 
neither gains any utility. If switching allocations results in 
each user being able to process the same number of jobs, 
then the users have either not changed their resource allo- 
cations or have gained resources; in the latter case, their 
utility decreases due to paying for extra resources that they 
do not use to process jobs. 

If users are not required to pay the difference in prices 
from switching resource allocations, then one user may gain 
utility by switching resource allocations with another: con- 
sider, for instance, a user who processes a very small amount 
of jobs, due to a low willingness to pay (e.g., a high value 
of Qj and low value of Cj in ((2])'s utility functions). Then 
this user will process a higher number of jobs by switching 
resources allocations with another user who is more willing 
to pay for resources in exchange for processing jobs. □ 

We next find that the fairness function (|14|) is Pareto-efficient 
if ^ > and |A| > - 1 [n [12]. Thus, if the allocation 



Uj (x'jyp) Pareto-dominates the allocation Uj (x* 



then 



□ 



the fairness function (|15p . for which A — 1/13 — 1, attains 
a larger value under the prices p than it does under the 
prices q. The prices here may represent those under any 
pricing plan; bundled, resource, or differentiated. If Lemma 
[3] holds and the utility received from each user j is equal 
to a price-independent constant multiplied by the amount 
paid by user j, then the revenue ((9]) is also Pareto-efficient, 
as it is an affine transformation of the sum of the utilities 
received. This latter quantity corresponds to the fairness 
(fT4|l as A — cx). 



